1. Field of the Invention
The present invention relates to an image pickup apparatus and an image processing method of obtaining an image with a broad dynamic range.
2. Description of the Related Art
[Prior Art]
Dynamic range of image sensor such as CCD used in an image pickup device such as widely prevailing digital cameras are generally narrower than those of film. Hence, in the case of imaging imaging a high luminance subject, the amount of received light exceeds the dynamic range. Then, the output of the image sensor saturates to cause the missing of the information of the subject imaged.
When, for example, an indoor scene is picked up by an imaging apparatus such as digital still cameras, it sometimes occurs that subjects present in the interior can be imaged well, but the blue sky observed through a window is imaged with saturation. The image picked up in the situation is unnatural as a whole. Such problem comes from the fact that the dynamic range of an image is narrow. To solve the problem, dynamic range is expanded by combining two images picked up separately.
For example, short time exposure image (low sensitivity image) is firstly picked up with high shutter speed, then long time exposure image (high sensitivity image) is successively picked up with low shutter speed. These two images are combined to superimpose the outdoor scene through the window in the low sensitivity image on the high sensitivity image in which the indoor scene is preferably picked up.
In a conventional technique described in JP-A-2000-307963, as the part of moving subjects do not perfectly coincide with low sensitivity image and high sensitivity image when the two images is combined, the high sensitivity image is partly replaced by the low sensitivity by using a mask.
In the above-described conventional technique of JP-A-2000-307963, although image signals of the two images are combined with use of the mask, the discrepancy of white balance is not considered. Therefore, as a result of that white balances of the high sensitivity image and the low sensitivity image in the combined image are respectively different, the combined image comes to be unnatural depending on an imaging scene.
Recently, an imaging apparatus such as digital cameras, which includes a new image pickup device having both of high sensitivity pixel and low sensitivity pixel, is proposed. The imaging apparatus combines a high sensitivity image (which will be also called “high output image” hereinafter) picked up by the high sensitivity pixel and a low sensitivity image (which will be also called “low output image” hereinafter) picked up by the low sensitivity pixel to output data of a single image. It is necessary to solve the problem associated with the above-described, when the image pickup apparatus combines images.
In the case of imaging a very bright subject under a high-contrast imaging condition, the image combination according to the conventional technique of JP-A-2000-307963 is effective in the reproduction of details in bright (highlight) portions, since the dynamic range is expanded. However, in the case of imaging under a low-contrast imaging condition such as a cloudiness or an interior, similar preferable effects are hardly achieved, or rather, the tone reproduction is wasted.
FIG. 18 shows the relation between a high sensitivity signal obtained by imaging a scene with a digital camera and the combined signal derived by combining the high sensitivity signal with a low sensitivity signal by a conventional technique. This figure depicts data associated with the signal of any one color among R (red), G (green) and B (blue). The ordinate indicates tone value, while the abscissa indicates subject luminance. The thin line in the figure is the high sensitivity signal prior to combination, and the thick line is the combined signal.
As shown in the figure, the region from a to b of the tone value of the high sensitivity signal (indicated by the hatched zone) is not utilized depending on the luminance of the scene by the combination operation. Therefore, there exist cases where it would be rather better that the image combination is not performed under a low-contrast imaging condition.
In another conventional technique described in JP-A-6-141229, the combination of two images is performed by converting each of the image signals obtained with low and high shutter through the same γ characteristics and then additively combining the both signals that are each converted with γ characteristics. However, since simple addition gives an image in which the middle tone region that is strongly influential on image quality appears unnatural, weighted addition depending on signal levels is usually carried out.
The conventional technique of JP-A-6-141229 intends to retain a preferable tone reproduction in the middle tone region through weighted addition of the image signals of the two images depending on signal levels and only the image signal with the high sensitivity is used for the middle tone region. However, even if a weighted addition depending on signal levels is performed, the white balance of the middle tone tends to differ from that of the highlights tone when each white balances of two images with high sensitivity and low sensitivity is not accurately adjusted. As a result, a combination image comes to be unnatural.